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REMARKS 



Claims 25, 29-31, 33, 37-39, and 41-50 are pending, with claims 29, 37, and 45 being 
independent. Claims 45-50 have been amended. No new matter has been added. 
Reconsideration and allowance of the above-referenced application are respectfully requested. 

Rejections Under 35 U.S.C. §112 

Claims 45-50 stand rejected under 35 U.S.C. § 1 12, second paragraph, as allegedly being 
indefinite. Without conceding the propriety of this rejection, claims 45-50 have been amended to 
obviate the rejection. Thus, withdrawal of the rejection under 35 U.S.C. §112, second paragraph, 
is respectfully requested. 

Rejections Under 35 U.S.C. $103 

Claims 25, 29-31, 33, 37-39, and 41-50 stand rejected under 35 U.S.C. § 103(a) as 
allegedly being unpatentable over Sawhney, H. E., et al., "Video Flashlights - Real Time 
Rendering of Multiple Videos for Immersive Model Visualization" (hereinafter "Sawhney"). 
This contention is respectfully traversed. 

The substance of the rejections in the 6-13-2008 Office Action are essentially the same as 
that presented in the 1 1-13-2007 Office Action (aside from the repositioning of text in light of 
the prior amendments). Thus, the 3-13-2008 Response filed in this application is hereby 
incorporated by reference. In addition, the Office's responses to the prior arguments are 
addressed. 

In the 3-13-2008 Response, it was noted that Sawhney does not described placing a 
surface that corresponds to the moving region in the three dimensional model and projecting 
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real-time video imagery information onto the three dimensional model, including the surface . 1 

In response, the Office states 2 : 

In response to Applicant's argument (on page 9 in first paragraph) that the reference 
Sawhney does not state the video information is projected on to such models, Contrarily, 
Sawhney teaches clearly in fig. 8d, see model red lines. On pages 9-10, Applicant argues that 
Sawhney does not describe, placing a surface that corresponds to a moving region in a three 
dimensional model, Examiner believes Sawhney e.g., in fig. 9 illustrates a moving region (a 
moving aerial) in a 3-D model e.g., using model lines in fig. 8d. 

However, Figure 8d in Sawhney shows model lines being projected on to the image using an 
estimated pose of the camera, not projecting real-time video imagery information onto a three 
dimensional model including a surface placed into the model to correspond to a moving region 
identified in the real-time video imagery information. 

Further, the reference to Figure 9 in Sawhney is not understood. Figure 9 in Sawhney 
shows the rendering of two frames of a video captured from a moving aerial autonomous 
helicopter. The suggestion by the Office the "moving aerial", which is the helicopter having a 
camera that captures the video imagery, can somehow be equated with the moving region 
identified within the video imagery itself completely defies common sense, and contradicts the 
other stated bases for rejecting the claims. With all due respect to the Office, nothing in the cited 
portions of Sawhney (or any other portion of Sawhney) teaches projecting real-time video 
imagery information onto a surface that has been placed into a three dimensional model of a 



1 See 3-13-2008 Response at page 9. 

2 See 6-13-2008 Office Action at page 2. 
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three dimensional environment in order to correspond to a region in motion identified within that 
very same real-time video imagery information. 

In addition, in the 3-13-2008 Response, in order to clearly distinguish over Sawhney, 
claim 29 was amended to recite, "wherein placing the surface comprises casting a ray from an 
optical center, corresponding to the real-time video imagery information, to a bottom point of the 
moving region in an image plane in the three dimensional model, and determining a position, an 
orientation and a size of the surface based on the ray, a ground plane in the three dimensional 
model, and the moving region." 3 In the 6-13-2008 Office Action, the exact same "Examiner's 
note" has been copied from the old rejection of claim 26 to the new rejection of claim 29, but the 
argument made regarding this Examiner's note has not been addressed. As pointed out before, it 
should be noted that the ray casting claimed relates to casting a vector in the three dimensional 
model 4 and does not refer to generating pixels on a display. Since the current rejection fails to 
address this point, there is a clear legal or factual deficiency in the current rejection. 

In the 3-13-2008 Response, it was also noted that nothing Figure 10 (of Sawhney) or its 
corresponding description says anything about determining a position, an orientation and a size 
of a surface that is placed in a three dimensional model, let alone "casting a ray from an optical 
center, corresponding to the real-time video imagery information, to a bottom point of the 
moving region in an image plane in the three dimensional model." In fact, the only technique 
described in Sawhney for modeling a moving object is to use "stereo depth information" from 



3 See e.g., Specification at paragraphs 101-106. 

4 See e.g., Specification at paragraph 104. 
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"stereo cameras." 5 This is in sharp contrast with the presently claimed techniques, which can be 
employed using only a single camera. 6 In response to this, the Office states 7 : 

In response to Applicant's argument on page 10 in second paragraph the reference's 
technique described a moving object uses stereo cameras, and this is in sharp contrast with the 
claimed invention, which can be employed using a single camera. Examiner's reply: in light of 
the specification e.g. in fig. 10 illustrates at least three cameras. 

Examiner's notes: Applicant my point out where is the limitation of using only a single 
camera in the claims? 

First, the reference to FIG. 10 of the present specification is completely irrelevant since FIG. 10 
of the present specification is not related to the claimed process of placing a surface into a three 
dimensional model; rather FIG. 16 relates to this claimed subject matter. Thus, the Office 
appears be defying well established law that the Office may not read limitations into the claims 
from the specification. 

Second, as to the actual language of the claim, it has not been argued that the claim 
recites only using a single camera. Rather, the claim recite, "wherein placing the surface 
comprises casting a ray from an optical center, corresponding to the real-time video imagery 
information, to a bottom point of the moving region in an image plane in the three dimensional 
model, and determining a position, an orientation and a size of the surface based on the ray, a 
ground plane in the three dimensional model, and the moving region." The specific single 
camera implementation (which is described in connection with FIG. 16 of the present 
application) has been noted only to assist the Office in understanding the plain meaning of the 

5 See Sawhney at sections 2 and 5, at pages 158 and 165. 

6 See e.g., Specification at FIG. 16 and corresponding description. 

7 See 6-13-2008 Office Action at page 2. 
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claim language. Sawhney teaches the modeling of a moving object using stereo depth 
information from stereo cameras, which does not in any way teach or suggest "placing a surface 
that corresponds to the moving region in the three dimensional model, wherein placing the 
surface comprises casting a ray from an optical center, corresponding to the real-time video 
imagery information, to a bottom point of the moving region in an image plane in the three 
dimensional model, and determining a position, an orientation and a size of the surface based on 
the ray, a ground plane in the three dimensional model, and the moving region." 

Thus, for at least the above reasons, independent claim 29 should be in condition for 
allowance. Similar arguments are applicable to claims 37 and 45, and thus, independent claims 
37 and 45 should each be in condition for allowance. Dependent claims 25, 30, 31, 33, 38, 39, 
41-44 and 46-50 should be patentable based on their respective base claims and the additional 
recitations they contain. 

For example, claims 41, 43 and 49 each recite, "wherein identifying a foreground object 
comprises identifying the foreground object in the subtracted real-time video imagery 
information using a histogram-based threshold and a noise filter." As noted in the 3-13-2008 
Response, the cited portion of Sawhney clearly describes the use of a threshold applied to a 
difference image between a current image and a median of a stack of images. 8 Sawhney does not 
indicate that the threshold is histogram-based or that a noise filter is used. In response, the 
Office refers to page 164 and Sawhney' s teaching that "the absolute value at each point can then 
be computed, and the result can be thresholded in order to highlight intensity or feature 



8 See Sawhney at Figure 10 and section 5, at page 164 and 167. 
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differences between the current video image and the reference image[. ]" 9 Thus, once again, the 
Office has failed to explain how this teaching of Sawhney is considered to teach the use of a 
histogram-based threshold. 

Also noted in the 3-13-2008 Response was that fact that Sawhney does not indicate that a 
noise filter is used in identifying a foreground object, and the Office's interpretation that "a noise 
filter" may be considered "as a virtual camera view in fig. 3" 10 is not understood and is without 
rational basis. In response, the Office refers to page 160 and Figure 3 of Sawhney, asserting that 
such "clearly illustrates final rendered composite that has been filtered the video pixels." 11 
Again, this doesn't address the claimed subject matter, which requires the use of a noise filter in 
the process of identifying a foreground object. Thus, for at least these additional reasons, there 
are clear legal or factual deficiencies in the current rejection of claims 41, 43 and 49. 

Claims 42, 44 and 50 each recite, "wherein identifying a region in motion in real time 
further comprises estimating the background image by modeling the background image as a 
temporal pixel average of five recent image frames in the real-time video imagery information." 
With respect to this subject matter, the Office "takes an official notice for an average of five 
recent frames, because Sawhney on page 161 under 'A pseudo-code for the rendering algorithm 
..' teaches as one of the parameters that a user may be indicated a number for number of frames 
'frame number', and that number may be 5 image frames." 12 This official notice is explicitly 
challenged. The "Frame Number" is a parameter to the "Update VideoContent" procedure, 
which clearly relates to retrieving the most recent available video data from a camera specified 

9 See 6-13-2008 Office Action at pages 2-3. 

10 See 1 1-13-2007 Office Action at page 4. 

11 See 6-13-2008 Office Action at page 3. 

12 See 6-13-2008 Office Action at page 8. 



Applicant : Ulrich Neumann et al. Attorney's Docket No.: 06666-156001 / #3345A 

Serial No. : 10/676,377 

Filed : September 30, 2003 

Page : 14 of 15 

by "Video Source". Contrary to the Office's assertion, nothing in Sawhney suggests that this 
procedure is part of the process of moving object detection or that the "Frame Number" may be 
an indication to use five recent image frames to generate a temporal pixel average to model the 
background image. 

Furthermore, taking official notice regarding this claimed subject matter is inappropriate 
since the Office has previously contended that "applicant has established criticality for the use of 
five frames in the averaging technique" and "optimizing such a number of frames would require 
undue experimentation under In re Wands, since applicant points out that five frames has a 
specific benefit, as per the specification." 13 In response to these points, the Office states 14 : 

wherein identifying a region in motion in real time further comprises estimating the background 
image by modeling the background image as a temporal pixel average of five recent image 
frames in the real-time video imagery information (e.g., on page 164 at right col. teaches the 
reference background image needs to be constantly updated during the day to reflect changing 
ambient illumination. Alternatively only moving objects may be detected by comparing current 
image with an image taken a few seconds before. The 2D moving object detection can be done in 
real time, also see page 162 left column at third paragraph). On the other hand, comparing fig. 15 
of the current invention with fig. 10 of the reference: in fig. 15 steps 1500, 1510, 1520, and 1530 
illustrate what is shown the left side of fig. 10, and fig. 10 illustrates the validated foreground 
object in step 1550 in the middle of fig. 10. 
With all due respect to the Office, the undersigned attorney completely fails to understand how 
this response actually addresses the arguments presented. Thus, there are clear legal or factual 



13 See 1 1-28-2005 Office Action at page 6. 

14 See 6-12-2008 Office Action at page 3. 
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deficiencies in the rejection of claims 42, 44 and 50, and these claims should be in condition for 
allowance. 



It is believed that all of the pending claims have been addressed. However, the absence 
of a reply to a specific issue or comment does not signify agreement with or concession of that 
issue or comment. Because the arguments made above may not be exhaustive, there may be 
reasons for patentability of any or all pending claims (or other claims) that have not been 
expressed. Finally, nothing in this paper should be construed as an intent to concede any issue 
with regard to any claim, except as specifically stated in this paper. 

It is respectfully suggested for all of these reasons, that the current rejections are 
overcome, that none of the cited art teaches or suggests the features which are claimed, and 
therefore that all of these claims should be in condition for allowance. A formal notice of 
allowance is thus respectfully requested. In the absence of such, a telephone interview is 
requested to review the claims and the cited art. 

No fees are believed due with this response. Nonetheless, please apply any necessary 
charges or credits, to deposit account 06-1050. 



Conclusion 



Respectfully submitted, 




William E. Hunter 
Reg. No. 47,671 
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